This commit is the start of a refactoring which will move from each code
generator having one monolithic test into many separate sub-tests. This
should enable smaller and more focused tests which benefits bringing new
bindings generators online (can get some tests working before others)
and also allows writing tests for some generators but not others.

The way that this is currently architected is that there's a new
`tests/runtime/*` directory in which a test-per-directory lives. For
each tests `tests/runtime/foo` there will be a few conventions:

* `tests/runtime/foo/imports.witx` - these are what hosts must
  implement and what the wasm can import from the host. This file is
  required to exist.
* `tests/runtime/foo/exports.witx` - this is what wasms must implement
  and what hosts can call. This file must exist.
* `tests/runtime/foo/host.*` - these are separate host implementations
  to execute the wasm code. The extension of the file indicates which
  generator it's used for (e.g. right now `host.{ts,rs,py}` are
  supported). If `host.py` doesn't exist for the test `foo` then testing
  the Python code generator will skip the test `foo`. Otherwise it's
  expected to work. At least one `host.*` file is required to work, but
  there's no need to have a file-per-host.
* `tests/runtime/foo/wasm.*` - these are separate guest compiled-to-wasm
  implementations. Currently only `wasm.{c,rs}` are supported, but the
  intent is to support `wasm.js` for #71 as well. Not all languages need
  to have an implementation here either, they'll be skipped if omitted.

The general goal is that adding a test for some witx functionality
should be "drop a few files in `tests/runtime/your_new_test` and you're
good to go. Over time more generators can add more tests if it becomes
necessary, and generator-specific tests can also be added to this
directory such as the one test exercising JS instantiation added here
too.

Overall I think this turned out quite well. The biggest downside is that
Rust-compiled-to-wasm needs to list explicitly each test that's being
compiled to wasm, which is a bit of a bummer.

Starts the extraction process from the monolithic tests that already
exist into smaller tests.

There was an accidentaly `.to_snake_case()` when it should have been
using the raw name from the witx.

Closes #76

This commit is the initial steps towards supporting and implementing
`async` function as-specified in `*.witx` itself. The goal of this is to
implement the callback-based ABI of async functions and streams as
presented in WASI subgroup meetings.

This commit itself only implements async functions for the
compiled-to-Rust guest and a JS host. The next PR is to implement async
functions for a Wasmtime host, followed by an implementation of streams
afterwards.

Included here is:

* Updates to the `*.witx` syntax and parsing.
* Assertions in code generators that don't support async that async
  isn't used.
* An implementation of the callback ABI for `async` functions for
  compiled-to-wasm Rust modules. This includes a simple "executor" to
  track the state of async imports/exports. Everything is only exposed
  with `async` functions in Rust, no callbacks are part of the interface
  of the generated bindings.
* An implementation of the callback ABI for JS hosts. This is
  exposed with `async` functions and only uses callbacks under-the-hood.

This change had some significant ramifications on the generated bindings
for JS hosts. Namely both imports and exports needed to access the same
state of callbacks so they are no longer generated separately in
isolation but rather together in the same manner as the spidermonkey
bindings. I expect that most generators will migrate to this pattern in
the future. Work was done internally to deduplicate intrinsics across
bindings modules, namely the promise-related intrinsics.

The testing so far isn't super strenuous, but a major thing I made sure
to implement is that when an asynchronous wasm function is called and
later some asynchronous import throws an exception that should carry the
exception to the original wasm export's promise in JS. This is intended
to map to basically what `async` and `await` feel like in JS normally.
Otherwise though there's codegen tests for flavorful types being used
in parameters/results for async functions (and this test is ignored in
most generators at the moment) and runtime tests for some concurrency.

* feat: initial work to make nullable fields use option `?`

* fix: unwrap nullable type and update test to use `?`

* fix: missing semicolon

* fix: refactor to return option when unwrapping nullables

* feat: refactor nullable methods to iface struct

* fix: fmt forgot to run it in root

* chore: refactor nullable methods back to js impl

I today started looking at updating this project to the new draft of the
canonical ABI specified at WebAssembly/component-model#23 and while
there's quite a few changes that need to happen the first thing that
struck me was that maintaining all of the support for old-style witx
files is pretty onerous. Especially the ABI-calculating code has lots of
special cases for various witx constructs and the Preview1 ABI, and I
wasn't really sure how to update this in many situations.

Overall the original purpose of the witx support was to prove out that
it's possible to support both the old witx abi and the new canonical ABI
in the same generator. The canonical ABI has changed significantly in
the meantime however and this doesn't necessarily make sense to do any
more. I think it would be best now to reevaluate at the point when WASI
is ready to switch to the component model what to do with the old witx
support. I no longer think that "build it in here" is the obvious
option. As this diff shows there's quite a bit of weight to carry the
old witx abis as this commit clocks in at nearly 7000 lines removed.

The specifics being dropped here are:

* Parsing support for `*.witx`
* Support for `Pointer` and `ConstPointer`
* Support for `WitxInstruction`
* Support for push/pull buffers

The push/pull buffer feature was never actually fully implemented, even
for Rust hosts they only kind-of worked. Most other generators never
even implemented support for them. Additionally support for other
`*.witx` constructs was somewhat spotty at best with very few tests.

My hope is that there are no existing users of this support. If there
are then I think it's best to re-evaluate how best to solve the scenario
on-hand.

* Rename f32/f64 to float32/float64

This commit renames the previously-known `f32` and `f64` interface types
to `float32` and `float64` to match the upstream specification.

* Fix a test

* Add support for params passed indirectly

This commit adds support for the `MAX_FLAT_PARAMS` variable in the
current canonical ABI draft. Argument lists which exceed a static number
of parameters are now always passed indirectly through memory. In export
this is dynamically allocated/freed and in imports this is statically
passed.

* Review comments

* Split out the `Option` and `Expected` types from `Variant`

This commit is like prior PRs to split out specializations of types into
their own AST type to avoid conflicting with the main type (in this case
`variant`). I originally thought these two types would be relatively
simple but this is probably one of the more complicated transitions, as
evidenced by the lines changed here. The main churn was that variants
already have a significant amount of code to support them and this is in
some places "duplicating" code for option/expected and in other cases
splitting what was already an if/else.

Overall I think that the generated code gets a little better since it's
clear when something is and `option` vs `expected` now rather than
trying to have everything shoehorned into one. Notably the C code
generator now generates descriptive fields like `bool is_some` or `bool
is_err` instead of a bland `uint8_t tag` with some comments about how to
use it.

* Remove `Variant::as_{option,expected}`

... as these are separate variants now.

* Review comments

* Split out a `Union` type from `Variant`

This commit splits out the union specialization from `variant`,
finalizing the transition to the new AST of types available in the
current component model draft. This is mostly like the prior
splittings-out where there's some duplication but it's relatively
localized and hopefully not too hard to follow.

* Fill out TODO

* Update enum representation

* Generate variant docs

* Fix demo

* wit-bindgen-rust: fix lowering and lifting of zero-length lists.

This commit fixes the Rust bindings generated for lifting and lowering of lists
to account for lists of zero-length.

It is undefined behavior to call `std::alloc::alloc` with a zero-sized layout.

Additionally, as the behavior of the `canonical_abi_realloc` implemented in
`wit-bindgen` is to return a non-null dummy pointer for a zero-sized alloc, the
bindings were calling `std::alloc::dealloc` with a pointer that was not
allocated with `std::alloc::alloc` (also undefined behavior).

To fix this, the bindings now account for zero-length lists and skip allocation
and deallocation where appropriate.

* Update `list` runtime tests to pass empty lists and strings.

* initial work

* Fix formatting

* Fix references to Rust library for guest code
Improve documentation
Other minor fixes

* Fix wastime -> host_wasmtime_rust references
Fixed feature flags

* Fix demo deploy
update readme
improve help text

* Update readme

Co-authored-by: Kyle Brown <kbrown@singlestore.com>
Co-authored-by: Kyle Brown <kyleb@liquidrocketry.com>

* Update JS integer lowering

Updates JS integer lowering to match the component model explainer (assuming that WebAssembly/component-model#52 is merged).

The main differences are that out-of-range numbers now wrap instead of throwing, and that non-integers are now rounded down instead of throwing.

* Remove `async func` support.

The `async` keyword has been removed from the [async proposal], in favor
of using plain functions that with `future` or `stream` types. See also
the [component-model PR to remove the syntax].

This removes the (now) old `async func` suppport from wit-bindgen, to make
it easier to develop the current Canonical ABI proposal.

[async proposal]: https://docs.google.com/presentation/d/1MNVOZ8hdofO3tI0szg_i-Yoy0N2QPU2C--LzVuoGSlE/edit#slide=id.g1270ef7d5b6_0_111
[component-model PR to remove the syntax]: WebAssembly/component-model#98

* Remove support for the `async` checkbox in wit-bindgen-demo.

* Update flag representation

* Use `flags.repr().count()` instead of manually matching

* Remove flag validation intrinsics

* Validate that flags don't have extraneous bits set when lifting

* grammar nit

Co-authored-by: Alex Crichton <alex@alexcrichton.com>

* Rename `canonical_abi_realloc` to `cabi_realloc`.

This follows the current Canonical ABI. This doesn't rename
`canonical_abi_free`, as that's expected to be removed when post-return
functions are implemented.

* Rename the "expected" type to "result".

This follows the current Canonical ABI.

* Implement function and value type name mangling.

This implements the name-mangling scheme in the current canonical ABI,
with the modification proposed in WebAssembly/component-model#104,
though that can be easily removed if the proposal is declined.

* Use name mangling in the bindings generators.

* Use the export base name rather than the mangled name for python identifiers.

Signed-off-by: GitHub <noreply@github.com>

Signed-off-by: GitHub <noreply@github.com>

* WIP: Update for component model multireturn and removal of unit

This PR updates wit-bindgen for the latest changes to the component model:

- the return of multireturn
- removal of the `unit` type
- a new syntax for optional types in results, streams, futures and variants

all of which go hand in hand.

This also pulls in the latest versions of wasm-encoder, wasmprinter,
and wasmparser crates to get their updates for these component model changes.

* Get all wit-bindgen tests working

* Implement tests for multi-return

Exercise this throughout the runtime tests and additionally add a few
variants in codegen tests to ensure that this is exercised.

* Update markdown generator

Co-authored-by: George Kulakowski <gkulakowski@fastly.com>

This adds a few new pseudo-instructions plus a new method on `Interface`
to generate a `post-return` function. For now these are simply named
`{name}_post_return` and the integration point there will probably
change as the component model shapes up.

The integration here is intended to still be relatively primitive in
that the actual component model integration will likely look different
in the future. Given how `wit-bindgen` works today, though, this should
at least get things part of the way there.

* remove Canonical ABI name mangling

this reverts a big chunk of the functionality introduced in bytecodealliance/wit-bindgen#309

* multi-return.wit: namespace these export functions

the tests already export functions named a,b,c in strings.wit. We can't
define more functions with that same name.

* remove mangled names from tests/runtime/invalid/wasm.rs

Try to reduce the number of places versions are mentioned and encourage
shared dependencies between crates.

…409)

* gen-host-js: Error wrapping and unwrapping for singular results

When an exported component function has a singular return of type
result, treat this as a throwable in the JS enviroment, by throwing
the error payload.

Similarly, when an imported function has a singular return of type
result, add a catch handler and wrap the function in a result object
when handling.

Further optimization work should be done to avoid unnecessary wrapping
in future.

* better payload check, return fixup

* fixup demo

* improved error payload extraction

* fixup tests, intrinsic dependence

* remove unnecessary tostring

* pr feedback

* remove unnecessary include

* simpler throws check

* review feedback

* explicit result flattening

* throws signature

* fixup throws signature

* Add empty `imports.wit` for `exports_only` test

Otherwise the build process thinks this is always missing so tests are
always slowly rebuilt.

* Fix compiling the `runtime` test for JS in isolation

This requires the `clap` feature to be enabled now which works with
`cargo test --workspace` but isn't working with individual crate
testing.

* Don't test wasm size in exports_only test

Different versions of `wasi-libc` will produce differently sized
binaries, so only test the generated JS which this repo controls.

…ining (#425)

In addition, --instantiation now supports a third instantiateCore optional argument to
replace the --load=custom wrapping use case

)

* Update all codegen tests to world syntax

This enables simplifying codegen tests as well by having each test
exercise imports/exports instead of having all code generators have one
case for imports and one for exports.

* Update codegen tests for the C guest generator

* Update teavm codegen tests for worlds

* Remove the `--name` argument from the CLI

No longer needed as the name is inferred from the `*.wit` world.

* Get all js host generator tests working

Update all `runtime/*` tests with new naming conventions, world files,
etc. Minor updates were made to names as I ended up using different
conventions for the `*.wit` world files than were previously exercised.

This also fixes a few minor issues with the output for the JS generator
in the default ESM mode.

* Get all host-wasmtime tests working with worlds

Various small updates here and there to namings and such.

* Get all Python tests working with worlds

Fix support for exported instances by using correct import paths for
various types/intrinsics/etc. Additionally update the codegen tests to
specify a mypy cache dir to avoid racing between tests.

* Update demo build for worlds

Also fix an issue with the Rust host generator where it used an
interface's name instead of the name of the import for import module names.

* Update demo with world files

* Bump version of dependency crates

This commit updates this repository to published versions of the
`wasm-tools` family of crates. Additionally Wasmtime is updated with the
same upstream versions of the `wasm-tools` crates. Put together this
updates everything to the most recent version of the component model
with optional URLs encoded and an updated version of Wasmtime.

Part of this transition is the removal of
`wit_component::ComponentInterfaces` which is plumbed through as part of
this commit.

* Update to wasmtime main

* Fix demo build

* Update wasm-tools binary

* Fix cli build

* Install wasm-tools from crates.io, not git